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Abstract 

We study a model of multi-excited random walk on a regular tree which generalizes 
the models of the once excited random walk and the digging random walk introduced 
by Volkov (2003). We show the existence of a phase transition and provide a crite- 
rion for the recurrence/transience property of the walk. In particular, we prove that 
the asymptotic behaviour of the walk depends on the order of the excitations, which 
contrasts with the one dimensional setting studied by Zerner (2005). We also consider 
the limiting speed of the walk in the transient regime and conjecture that it is not a 
monotonic function of the environment. 
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1 Introduction 

The model of the excited random walk on 1i d was introduced by Benjamini and Wilson in [6] 
and studied in details in, for instance, [U [3 HlJ [12j [201 [2T] . Roughly speaking, it describes 
a walk which receives a push in some specific direction each time it reaches a new vertex 
of Z d . Such a walk is recurrent for d — 1 and transient with linear speed for d > 2. In 
[24] [25] , Zerner introduced a generalization of this model called multi-excited random walk 
(or cookie random walk) where the walk receives a push, not only on its first visit to a 
site, but also on some subsequent visits. This model has received particular attention in the 
one-dimensional setting (c.f. [2j [4j [21 [lOl [17] and the references therein) and is relatively well 
understood. In particular, a one-dimensional multi-excited random walk can be recurrent 
or transient depending on the strength of the excitations and may exhibit sub-linear growth 
in the transient regime. 

Concerning multi-excited random walks in higher dimensions, not much is known when 
one allows the excitations provided to the walk to point in different directions. For instance, 
as remarked in [10], for d > 2, when the excitations of a 2-cookies random walk push the 
walk in opposite directions, then there is, so far, no known criterion for the direction of 
transience. In this paper, we consider a similar model where the state space of the walk 
is a regular tree and we allow the excitations to point in opposite directions. Even in this 
setting simpler than Z d , the walk exhibits a complicated phase transition concerning its 
recurrence/transience behaviour. 

Let us be a bit more precise about the model. We consider a rooted 6-ary tree T. At 
each vertex of the tree, we initially put a pile of M > 1 "cookies" with ordered strengths 
pt, ■ ■ ■ ,Pm S [0, 1). Let us also choose some other parameter q £ (0, 1) representing the bias 
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of the walk after excitation. Then, a cookie random walk on T is a nearest neighbor random 
walk X = (X n ) n >o, starting from the root of the tree and moving according to the following 
rules: 

• If X n = x and there remain the cookies with strengths Pj,Pj+i, ■ ■ ■ ,Pm at this vertex, 
then X eats the cookie with attached strength Pj and then jumps at time n + 1 to the 
father of x with probability 1 — Pj and to each son of x with probability Pj/b. 

• If X n — x and there is no remaining cookie at site x, then X jumps at time n + 1 to 
the father of x with probability 1 — q and to each son of x with probability q/b. 

In particular, the bias provided to the height process \X\ upon consuming a cookie of 
strength p is 2p— 1. Therefore, a cookie pushes the walk toward the root when p < 1/2 and 
towards infinity when p > 1/2. The main question we address in this paper is to investigate, 
whether X is recurrent or transient i.e. does it return infinitely often to the origin or does 
it wander to infinity. 

For the one dimensional cookie random walk, a remarkably simple criterion for the recur- 
rence of the walk was obtained by Zerner [24] and generalized in [10]. This characterization 
shows that the behavior of the walks depends only on the sum of the strengths of the cook- 
ies, but not on their respective positions in the pile. However, in the tree setting considered 
here, as in the multi-dimensional setting, the order of the cookies does matter, meaning that 
inverting the position of two cookies in the pile may affect the asymptotic behaviour of the 
walk. We give here a criterion for recurrence from which we derive explicit formulas for 
particular types of cookie environments. 

1.1 The model 

Let us now give a rigorous definition of the transition probabilities of the walk and set some 
notations. In the remainder of this paper, T will always denote a rooted 6-ary tree with 
b > 2. The root of the tree is denoted by o. Given x £T, let x stand for the father of x and 
x , x , . . . , x stand for the sons of x. We also use the notation |cc| to denote the height of 
a vertex x € T. For convenience, we also add an additional edge from the root to itself and 
adopt the convention that the father of the root is the root itself ( o = 6). 

We call cookie environment a vector C = (pi,P2, ■ ■ ■ ,PM ; q) € [0, 1) M x (0,1), where 
M > 1 is the number of cookies. We put a semicolon before the last component of the 
vector to emphasize the particular role played by q. AC multi-excited (or cookie) random 
walk is a stochastic process X — (X n ) n >o defined on some probability space (f2,.F, P), 
taking values in T with transition probabilities given by 

P{X = o} = l, 

P{-^n+l = X n | Xq, 
P{^n+1 = X n | Xq, 

where i S {1, . . . , b} and j = jj{0 < k < n, Xk = X n } is the number of previous visits of the 
walk to its present position. 

Remark 1.1. 1. We do not allow q — in the definition of a cookie environment. This 
assumption is made to insure that a — 1 law holds for the walk. Yet, the method 
developed in this paper also enables to treat the case q = 0, c.f. Remark 18. II 

2. When pi = P2 = ■ ■ ■ = Pm = q, then X is a classical random walk on T and its height 
process is a drifted random walk on Z. Therefore, the walk is recurrent for q < i and 
transient for q > \. More generally, an easy coupling argument shows that, when all 



f ifj<M, 
f if j>M, 

1 - P 3 if j < M, 
l-q if j>M, 
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the pi's and q are smaller than | (resp. larger than |), the walk is recurrent (resp. 
transient). The interesting cases occur when at least one of the cookies pushes the 
walk in a direction opposite to the bias q of the walk after excitation. 




3. This model was previously considered by Volkov [23] for the particular cookie environ- 




(b) (0, ; £rj) "two-digging random walk". 

In both cases, Volkov proved that the walk is transient with linear speed and conjec- 
tured that, more generally, any cookie random walk which moves, after excitation, like 
a simple random walk on the tree (i.e. q = b/(b+ 1)) is transient. Theorem 11.21 below 
shows that such is indeed the case. 

Theorem 1.2 (Recurrence/Transience criterion). 

Let C = (pi,p2, ■ ■ ■ ,Pm \q) be a cookie environment and let P(C) denote its associated cookie 
environment matrix as in Definition \3.1\ This matrix has only a finite number of irreducible 
classes. Let A(C) denote the largest spectral radius of theses irreducible sub-matrices (in the 
sense of Definition 1 5. 1\) . 

(a) Lf q < and A(C) < then the walk in the cookie environment C is recurrent i.e. it 
hits any vertex o/T infinitely often with probability 1. Furthermore, if A(C) < \, then 
the walk is positive recurrent i.e. all the return times to the root have finite expectation. 

(b) Lf q > j^j- or X(C) > |, then the walk is transient i.e. lim„^oo \X n \ = +oo. 

Moreover, if C = (fii,p~2, ■ ■ ■ ,Pm ! 0) denotes another cookie environment such that C < C 
for the canonical partial order, then the C cookie random walk is transient whenever the C 
cookie random walk is transient. Conversely, if the C cookie random walk is recurrent, then 
so is the C cookie random walk. 

The matrix P(C) of the theorem is explicit. Its coefficients can be expressed as a rational 
function of the p^s and q and its irreducible classes are described in Section |4~T1 However, 
we do not know, except in particular cases, a simple formula for the spectral radius A(C). 

Let us stress that the condition A(C) < \ does not, by itself, insure the recurrence of the 
walk. Indeed, when X a biased random walk on the tree (pi = . . . = pm = q), then P{C) 
is the transition matrix of a Galton- Watson process with geometric reproduction law with 
parameter +b %_ ■ According to [19], we have 



Therefore, for q sufficiently close to 1, the walk is transient yet A(C) < 1/b. 

Let us also remark that the monotonicity property of the walk with respect to the initial 
cookie environment stated in Theorem ll.2[ although being quite natural, is not straightfor- 
ward since there is no simple way to couple two walks with different cookie environments (in 
fact, we suspect that such a coupling does not exist in general, see the conjecture concerning 
the monotonicity of the speed below). 

Theorem 1.3 (Speed and CLT when p ; > 0). 

LetC = (pi,p2, ■ ■ ■ ,Pm i <?) be a cookie environment such thatpi > for alii. If the C- cookie 
random walk is transient, then it has a positive speed and a central limit theorem holds: there 
exist deterministic v = v(C) > and a — <j(C) > such that 
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( a ) (piib+i) "once-excited random walk". 




— nv 



AA(0,a 2 ). 




v 



and 




it 



3 



v(Pi) 

1.0 --- 



0.8 - 



0.6 - 

0.4 - 



0.2 







1 1 1 1 k vi 

0.2 0.4 0.6 0.8 1.0 
Figure 1: Speed of a (pi, 0.01 ; 0.95) cookie random walk on a binary tree obtained by Monte 
Carlo simulation. 



The assumption that all cookies have positive strength cannot be removed. When some 
cookies have zero strength, it is possible to construct a transient walk with sub-linear growth, 
c.f. Proposition [L9l 

A natural question to address is the monotonicity of the speed. It is known that the 
speed of a one-dimensional cookie random walk is non decreasing with respect to the cookie 
environment. However, numerical simulations suggest that such is not the case for the model 
considered here (c.f. Figure [1]). We believe this behaviour to be somewhat similar to that 
observed for a biased random walk on a Galton- Watson tree: the slowdown of the walk is 
due to the creation of "traps" where the walk spends a long time. When p2 — 0, this is 
easily understood by the following heuristic argument: the walk returns to each visited site 
at least once (except on the boundary of its trace) and the length of an excursion of the 
walk away from the set of vertices it has already visited is a geometric random variable with 
parameter p\ (the first time the walk moves a step towards the root, it moves back all the 
way until it reaches a vertex visited at least twice). Therefore, as p\ increases to 1, the 
expectation of the length of theses excursions goes to infinity so we can expect the speed of 
the walk to go to 0. What we find more surprising is that this slowdown also seems to hold 
true, to some extend, when P2 is not zero, contrarily to the conjecture that the speed of a 
biased random walk on a Galton- Watson tree with no leaf is monotonic, c.f. Question 2.1 

of [u. 

1.2 Special cookie environments 

The value of the critical parameter X(C) can be explicitly computed in some cases of interest. 
Theorem 1.4. Let C — (pi,... ,pm ', q) denote a cookie environment such that 

p % = for all i < lM/2\ (1) 
where [x\ denotes the integer part ofx. Define 

^ n (u - «> to) + ^ + f to)" 1 ) ■ 

For q < £rj , it holds that 

A(C) = A. vm (C). 
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Figure 2: Phase transition of a (p ; q) cookie random walk on a binary tree. 



Remark 1.5. For any cookie environment, we have A(C) < 1 (it is the maximal spectral 
radius of sub-stochastic matrices). Moreover, when [M/2\ cookies have strength 0, the 
function q i— > A sym (pi, . . . ,pm ; q) is strictly increasing and A sym (pi, . . . ,pm ', gxj) = 1- Thus, 
A(C) < 1 < A sym (C) for all q > 

Let us also note that, under Assumption ([I]), in order to reach some vertex x, the walk 
has to visit every vertex on the path [o, a;) at least M times. Therefore, for such a walk, 
except on the boundary of its trace, every vertex of the tree is visited either or more than 
M times. This justifies A(C) being, in this case, a symmetric function of the p^s. 

The combination of Theorem \1.2\ Theorem 11.41 and Remark 11.51 directly yields particu- 
larly simple criterions for the model of the once excited and the digging random walk. 

Corollary 1.6 (Once excited random walk). 

Let X denote a (p;q) cookie random walk (i.e. M — 1) and define 

Then X is recurrent if and only if X\ < 1/b. 

In particular, the phase transition of the once excited random walk is non trivial in both 
cases p < \ < q and q < h < p (c.f. Figure [2|). 

Corollary 1.7 (M-digging random walk). 

Let X denote a C — (0, . . . , ; q) cookie random walk and define 

M times 

/ \ M+l 

A - = [kTT) ) ■ 

Then X is recurrent if and only if X dig < 1/b. 

Recall that, according to Theorem ll.2[ the condition q > b/(b+ 1) is sufficient to insure 
the transience of the walk. Corollarv l 1 . 71 shows that this condition is also necessary to insure 
transience independently of pi, . . . ,Pm'- for any q < b/(b+ 1), the M digging random walk 
is recurrent when M is chosen large enough. 
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We now consider another class of cookie environment to show that, contrarily to the one 
dimensional case, the order of the cookies in the pile does matter in general. 

Proposition 1.8. Let X be a C = (f>i,£>2, 0, . . . , ; q) cookie random walk with K > 2. 

K times 

Define v{pi,P2) to be the largest positive eigenvalue of the matrix 

Pi I P1P2 _ 2pip2 P1P2 

b "r b „ 6 2 ~TP~ 

P1+P2 _ 2pip 2 P1P2 

b 6 2 ~TP~ 



namely 



1 



f(Pi,Pa) = ^ ( (b-typuto + bp! + ^(b 2 -6b+l)plp% + 2b{b-l)p 2 lP2 + b 2 P \ + 4b Pl p% 

Recall the definition o/A Bym (C) given in Theorem \1.4\ and set 

A = max(A. ym (C),i/(pi,p 2 ))- 
The walk X is recurrent if and only if A < \ . 

Since v is not symmetric in (pi,Pi), Proposition 11.81 confirms that it is possible to con- 
struct a recurrent cookie random walk such that the inversion of the first two cookies yields 
a transient random walk. For 6 = 2, one can choose, for example, p% = |, p% = | and q < \. 

Proposition 11.81 also enables to construct a transient cookie random walk with sub- linear 
growth. 

Proposition 1.9. Let X be a C — (pi,P2, 0, ; q) cookie random walk with q > b/(b+ 1) 
and v(p\,p2) = l/b. Then X is transient yet 

liminfS = 0. 

n— >oo n 

We do not know whether the liminf above is, in fact, a limit. 



The remainder of this paper is organized as follows. In the next section, we prove a — 1 
law for the cookie random walk. In section 3, we introduce a branching Markov chain L (or 
equivalently a multi-type branching process with infinitely many types) associated with the 
local time of the walk. We show that the walk is recurrent if and only if this process dies out 
almost surely. We also prove some monotonicity properties of the process L which imply 
the monotonicity property of the cookie random walk stated in Theorem 11.21 In section 
4, we study the decomposition of the transition matrix P of L and provide some results 
concerning the evolution of a tagged particle. Section 5 is devoted to completing the proof 
of Theorem 11.21 In section 6, we prove the law of large number and C.L.T. of Theorem 11.31 
and Proposition 11.91 In section [3 we compute the value of the critical parameter A(C) for 
the special cookie environments mentioned above and prove Theorem 11.41 and Proposition 
11.81 Finally, in the last section, we discuss some possible extensions of the model. 
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2 The - 1 law 



In the remainder of the paper, X will always denote a C = (pi, . . . , Pm ', q) cookie random 
walk on a 6-ary tree T. We denote by T x the sub-tree of T rooted at x. For n £ IN, we also 
use the notation T„ (resp. T<„, T<„) to denote the set of vertices which are at height n 
(resp. at height < n and < n) from the root . We introduce the sequence (r„ )fc>o of return 
times to the root. 

r o d = f 0, 

r fe+1 d = f min{*>r fc , X t = o}, 

with the convention min{0} = oo. The following result shows that, although a cookie 
random walk is not a Markov process, a — 1 law holds (recall that we assume q ^ in the 
definition of a cookie environment) . 

Lemma 2.1 (0—1 law). Let X be a C cookie random walk. 

1. If there exists k > 1 such that P{t£ = oo} > 0, then lim^^oo \X n \ = oo P-a.s. 

2. Otherwise, the walk visits any vertex infinitely often P-a.s. 

Proof. Let us first assume that P{t£ < oo} = 1 for all k i.e. the walk returns infinitely often 
to the origin almost surely. Since there are no cookies left after the M th visit of the root, 
the walk will visit every vertex of height 1 infinitely often with probability 1. By induction, 
we conclude that the walk visits every vertex of T infinitely often almost surely. 

We now prove the transience part of the proposition. We assume that P{t^° < oo} < 1 
for some ko € IN. Let fli denote the event 

Oi = { lim \Xi\ = ooV. 

Given N e IN, let X N denote a multi-excited random walk on T reflected at height N (i.e. 
a process with the same transition rule as X but which always goes back to its father when 
it reaches a vertex of height N). This process takes values in the finite state space T<at and 
thus visits any site of TT<at infinitely often almost surely. For x € T<at, let f£° be the time 
of the fcg h return of X N to the vertex x. For n < N, let also f*° = sup xeTn f* be the first 
time when all the vertices of height n have been visited at least ko times. We consider the 
family of events (A„jv) n <Ar defined by: 

Ai,jv =' {^^does not reach height N before f^ }. 

Let us note that, on A ny jy, the processes X and X N are equal up to time f„°. Moreover, 
given neK and w g fli, we can always find N > n such that u £ A n ^. Hence, 

0i c n u An ' N - 

n>l N>n 

In particular, for any fixed n > 1, we get 

P{fii} < sup P{A„,at}. (2) 

N>n 

It remains to bound P{A nt N}- For x £ T„, we consider the subsets of indices: 

I x d ^ {0<i<^°, X? £T X }. 

I' x d = f {0<i<?*°, X? £T*}CI X . 
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With these notations, we have 

P{A n , N } = P{Vx S T n , (X-V S I x ) does not reach height N} 

< P{Vx G T„, (Xf, i G I' x ) does not reach height N}. 

Since the multi-excited random walk evolves independently in distinct subtrees, up to a 
translation, the stochastic processes (X^,i S I' x )xeT n are i.i.d. and have the law of the 
multi-excited random walk X starting from the root o, reflected at height N — n and killed 
at its k*Q return to the root. Thus, 

P{A„,jv} < P{(Xf"", i < fo°) does not reach height N - nV < P{r fe ° < oo} 6 ". (3) 
Putting ([2]) and together, we conclude that 

P{fM < P{r*° < oo} b " 
and we complete the proof of the lemma by letting n tend to infinity. □ 



3 The branching Markov chain L 
3.1 Construction of L 

In this section, we construct a branching Markov chain which coincides with the local time 
process of the walk in the recurrent setting and show that the survival of this process 
characterizes the transience of the walk. 

Recall that X N denotes the cookie random walk X reflected at height N. Fix fc > 0. 
Let CTfc denote the time of the fcg h crossing of the edge joining the root of the tree to itself: 



cr feo = inf ji > 0, 1 {x?=xf_ 1 =o} = k o}- 



£ N (x) M (t{0 < i < a kQ , If =x and X? +1 = x}. for all x G T</ 
We consider the (N + l)-step process L N = (Lq , , . . . , L 1 ^) where 



j'=i 

Since the reflected walk X N returns to the root infinitely often, we have a ko < oo almost 
surely. Let now £(x) denote the number of jumps of X N from x to x before time a ka i.e. 

J N 

Ll d = f (£ N (x), x G T n ) G M T " . 

Since the quantities L N , £ N depend on fco, we should rigourously write L N,ka , £ N > k °. Sim- 
ilarly, we should write <j^ q instead of cxfc . Yet, in the whole paper, for the sake of clarity, 
as we try to keep the notations as simple as possible, we only add a subscript to emphasize 
the dependency upon some parameter when we feel that it is really necessary. In particular, 
the dependency upon the cookie environment C is usually implicit. 

The process L N is Markovian, in order to compute its transition probabilities we need 
to introduce some notations which we will extensively use in the rest of the paper. 

Definition 3.1. 

• Given a cookie environment C = (pi, ■ ■ ■ ,Pm ',Q), we denote by (£i)i>i a sequence of 
independent random variables taking values in {0, 1, . . . , b}, with distribution: 



P{6 = 0} 



pte=i}=...=pfe=&} = { | 

We say that £j is a "failure" when £i = 0. 



I- Pi ifi<M, 
l-q ifi> M, 

El 
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• We call "cookie environment matrix" the non-negative matrix P — (p(i,j))i,j>a whose 
coefficients are given by p(0,j) = 1{j=o} an d, for i>l, 

7i n 

P (i,j) = P{ 1 fe=l} = •?'} where H = inf { n : H 1 {?,=0} = 
k=l k=l 

Thus, p(i,j) is the probability that there are exactly j random variables taking value 1 
before the i th failure in the sequence (£i,£2, ■ ■ ■)• 



The following lemma characterizes the law of L N . 

Lemma 3.2. The process L N = (Lq , , . . . , L^) is a Markov process on U^Li -"^ T " • ^ s 
transition probabilities can be described as follows: 

(a) Lq = (k ) i.e. £(o) = k Q . 

(b) For 1 < n < N and X\, . . . , x k £ T n with distinct fathers, conditionally on the 
random variables £ N (xi), . . . ,£ N (xk) are independent. 

(c) For x € T„ with children x e , the law of (t N {x ),..., £ N (x )), conditionally 
on , depends only on £ N (x) and is given by: 

p{{ N (x ') = 0, . . . ,£ N (x b ) = I i N (x) = o} = 1 
p{£ N (x l ) =i x , .. . ^(x") = | ^(a) - jo > 0} 

= Pjv/c € [0, 6], J({1 < i < j + . . . + j b , ^ = k) = ]k and &□+...+* = o}. 

In particular, conditionally on £ N (x) = jo, the random variable £ N (x ) is distributed 
as the number of £j 's taking value k before the j h failure. By symmetry, this distribu- 
tion does not depend on k and, with the notation of Definition \3.1\ we have 



{e N (x k ) = j\ e N (x)=jo}= P (jo,j). 



Proof, (a) is a direct consequence of the definition of Ofc . Let x £ T<at. Since the walk 
X N is at the root of the tree at times and <Jk , the number of jumps £ N (x) from x to x is 
equal to the number of jumps from x to x . Moreover, the walk can only enter and leave the 
subtree TPnT^jv by crossing the edge (x, x). Therefore, conditionally on £ N (x), the families 
of random variables (£ N (y), y £ T x n T< N ) and (£ N {y), y £ T< N \T X ) are independent. 
This fact implies (b) and the Markov property of L. Finally, (c) follows readily from the 
definition of the transition probabilities of a cookie random walk and the construction of the 
sequence (£i)i>i in terms of the same cookie environment. □ 

In view of the previous lemma, it is clear that for all x £ T<jv, the distribution of the 
random variables £ N (x) does not, in fact, depend on N. More precisely, for all N' > N, 
the (N + 1) first steps (L$ ,...,L%) of the process L have the same distribution as 
(Lq , . . . , Ljv). Therefore, we can consider a Markov process L on the state space U^Li : 

L = (L n , n > 0) with L n = (t(x), x£T n ) £ M T " 

where, for each AT, the family (£(x), x £ T<n) is distributed as (£ N (x), x £ T<jv). We can 
interpret L as a branching Markov chain (or equivalently a multi-type branching process 
with infinitely many types) where the particles alive at time n are indexed by the vertices 
ofT„: 
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• The process starts at time with one particle o located at £(o) = feo- 

• At time n, there are b n particles in the system indexed by T n . The position (in IN) of 
a particle x is £(x). 

• At time n + 1, each particle x £ T„ evolves independently: it splits into b particles 
x ,...,x . The positions £(x ),...,£( x ) of these new particles, conditionally on 
£{x), are given by the transition kernel described in (c) of the previous lemma. 

Remark 3.3. 

(1) Changing the value of k only affects the position l{o) of the initial particle but does 
not change the transition probabilities of the Markov process L. Thus, we shall denote 
by Pfc the probability where the process L starts from one particle located at £{o) = k. 
The notation will be used for the expectation under P&. 

(2) The state is absorbing for the branching Markov chain L: if a particle is at 0, then all 
its descendants remain at (if the walk never crosses an edge (x,x), then, a fortiori, 
it never crosses any edge of the subtree T x ). 

(3) Let us stress that, given £{x), the positions of the b children £{x ), . . . ,£(x ) are not 
independent. However, for two distinct particles, the evolution of their progeny is 
independent c.f. (b) of Lemma 13.21 

(4) When the cookie random walk X is recurrent, the process L coincides with the local 
time process of the walk and one can directly construct L from X without reflecting 
the walk at height N and taking the limit. However, when the walk is transient, one 
cannot directly construct L with N = oo. In this case, the local time process of the 
walk, stopped at its fcg h jump from the root to itself (possibly oo), is not a Markov 
process. 

Since is an absorbing state for the Markov process L, we say that L dies out when there 
exists a time such that all the particles are at 0. The following proposition characterizes the 
transience of the cookie random walk in terms of the survival of L. 

Proposition 3.4. The cookie random walk is recurrent if and only if, for any choice of k, 
the process L, under Pfc (i.e. starting from one particle located at £{o) = k), dies out almost 
surely. 

Proof. Let us assume that, for any k, the process L starting from k dies out almost surely. 
Then, k being fixed, we can find N large enough such that L dies out before time N with 
probability c arbitrarily close to 1. Looking at the definition of L, this means that the walk 
X crosses at least k times the edge (o, o) before reaching level N with probability c. Letting 
c tend to 1, we conclude that X returns to the root at least k times almost surely. Thus, 
the walk is recurrent. 

Conversely, if, for some k, the process L starting from k has probability c > never 
to die out, then the walk X crosses the edge (o, o) less than k times with probability c. 
This implies that X returns to the root only a finite number of times with strictly positive 
probability. According to Lemma [2~Tj the walk is transient. □ 

Recall that, in the definition of a cookie environment, we do not allow the strengths of 
the cookies pi to be equal to 1. This assumption insures that, for a particle x located at 
£(x) > M, the distribution (£(x ),..., £(x )) of the position of its b children has a positive 
density everywhere on M fc . Indeed, for any ji,...,j n £ M, the probability 

Pj^ 1 ) = ji, ■ ■ -^(x") = Jb I i(x) = i > Af} (4) 
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is larger that the probability of the i+ji + .-.+jb first terms of the sequence i being 



o,...,o,i,...,i,..., &,o 

i-l times ji times j b times 

which is non zero. Therefore, we get the simpler criterion: 

Corollary 3.5. The cookie random walk is recurrent if and only if L under Pm+i dies out 
almost surely. 



3.2 Monotonicity property of L 

The particular structure of the transition probabilities of L in terms of successes and failures 
in the sequence (£&) yields useful monotonicity properties for this process. 

Given two branching Markov chains L and L, we say that L is stochastically dominated 
by L if we can construct both processes on the same probability space in such way that 

£{x) < £{x) for all ieT, almost surely. 

Proposition 3.6 (monotonicity w.r.t. the initial position). For any < i < j, the 

process L under Pi is stochastically dominated by L under Pj . 

Proof. Since each particle in L reproduces independently, we just need to prove that L\ = 
(£{o ), . . . ,£(o )) under Pi is stochastically dominated by L\ under Pj and the result will 
follows by induction. Recalling that, under Pi (resp. Pj), £{o ) is given by the number of 
random variables £ taking value k before the z th failure (resp. j th failure) in the sequence 
(£„), we conclude that, when i < j, we can indeed create such a coupling by using the same 
sequence (£ n ) for both processes. □ 

Proposition 3.7 (monotonicity w.r.t. the cookie environment). 

Let C = (pi, . . . ,pm ; <?) and C — {pi,... ,Pm <j) denote two cookies environments such that 
C < C for the canonical partial order. Let L (resp. L) denote the branching Markov chain 
associated with the cookie environment C (resp. C). Then, for any i > 0, under Pi, the 
process L stochastically dominates L. 

Proof. Keeping in mind Proposition 13.61 and using again an induction argument, we just 
need to prove the result for the first step of the process i.e. prove that we can construct Li 
and Li such that, under Pi, 

e(o k ) < £{o k ) for all k € {1,...,6}. (5) 

Let (£„) denote a sequence of random variables as in Definition l3.1l associated with the cookie 
environment C. Similarly, let (£ n ) denote a sequence associated with C. When C < C, we 
have P{£„ = 0} > P{£ n = 0} and P{£„ = k} < P{£„ = k} for all k £ {1, . . . , b}. Moreover, 
the random variables (£ n ) n >i (resp. (£n)n>i) are independent. Thus, we can construct the 
two sequences (£„) and (£„) on the same probability space in such way that for all n > 1 
and all k € {1, . . . , b}, 

| w = implies £„ = 0, 
= k implies £„ = k. 

Defining now, for each k, the random variable £{o ) (resp. £{o )) to be the number of 
random variables taking value k in the sequence (£„) (resp. (£n)) before the i th failure, it is 
clear that ((5j) holds. □ 
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The monotonicity of the recurrence/transience behaviour of the cookie walk with re- 
spect to the initial cookie environment stated in Theorem 11.21 now follows directly from the 
combination of Corollary 13.51 and Proposition 13.71 

Corollary 3.8. Let C = (px,P2i ■ • ■ >Pm q) and C = (px,p~2, ■ ■ ■ ,Pm ', q) denote two cookie 
environments such that C <C. The C cookie random walk is transient whenever the C cookie 
random walk is transient. Conversely, if the C cookie random walk is recurrent, then so is 
the C cookie random walk. 

4 The Matrix P and the process Z 
4.1 Irreducible classes of P 

The matrix P plays a key role in the study of L. Since we allow the strength of a cookie to be 
zero, the transition matrix P need not be irreducible (a matrix is said to be irreducible if, for 
any there exists n such that p^ n \i,j) > 0, where p^ n \i,j) denotes the coefficient 
of P n ). 

For i,j £ IN, we use the classical notations 

• i — > j ]£p( n '(i,j) > for some n > 1. 

• i <-> j if i — > j and j — > i. 
Lemma 4.1. For any i,j S IN, we have 

(a) If p(i,j) > then p(i, k) > for all k < j and p(k,j) > for all k > i. 

(b) Ifi—*j then i — > k for all k < j and k — > j for all k > i. 

Proof. Recall the specific form of the coefficients of P: p(i,j) is the probability of having j 
times 1 in the sequence (£ n ) n >i before the z th failure. Let us also note that we can always 
transform a realization of (£ n )n>l contributing to p(i,j) into a realization contributing to 
p(i, k) for k < j (resp. for p(k,j) for k > i) by inserting additional failures in the sequence. 
Since no cookie has strength 1, for any n > 1, P{£ n = 0} > 0. Therefore, adding a 
finite number of failures still yields, when p(i,j) > 0, a positive probability for these new 
realizations of the sequence (£ n )- This entails (a). 

We have i — * j if and only if there exists a path i = no,ni,... ,n m -\,n m = j such 
that p(n t -i,nt) > 0. Using (a), we also have, for k < j, p{n m -i,k) > (resp. for k > i, 
p(k, Tlx) > 0). Hence i, m, . . . , n m -i, k (resp. k, m, . . . , n m -i,j) is a path from i to k (resp. 
from k to j). This proves (b). □ 

Lemma 4.2. Let a <b such that a ^ b. The finite sub-matrix (p(i, j)) a <i,j<b * s irreducible. 

Proof. Let i,j G [a,b]. In view of (b) of Lemma [4.1} a — » b implies i — > b and a — ► j. 
Therefore i — » b — > a — > j so that Thus, there exists a path in IN: 

i = n ,ni, . . . ,n m = j (6) 

such that p(n t -i,n t ) > for all t. It remains to show that this path may be chosen in [a, b]. 
We separate the two cases i < j and i > j. 

Case i < j. In this case, the path (J6J from i to j may be chosen non decreasing (i.e. 
nt-i < n t ). Indeed, if there exists < t < m such that nt-x > n t , then, according to (a) of 
Lemma \4. 11 p(n t ,n t +x) > implies that p(n t -x,n t +x) > 0. Therefore, n t can be removed 
from the path. Concerning the last index, note that, if n m _i > n m , then we can remove 
n m -x from the path since p(n m _2, n m ) > 0. 

Case i > j. According to the previous case, there exists a non decreasing path from i 
to i. This implies p(i,i) > and therefore p(i,j) > whenever j < i. Thus, there exists a 
path (of length 1) from i to j contained in [a, b}. □ 
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We now define 



{i > 0, p(i, i) > 0} = {i > 0, i <-> i}. 



On I, the relation <-> is an equivalence relation. In view of the previous lemma, we see that 
the equivalence classes for this relation must be intervals of IN. Note that {0} is always 
an equivalence class since is absorbent. Moreover, we have already noticed that, for 
hj > M + 1, p(i,j) > c.f. (01). Therefore, there is exactly one infinite class of the form 
[a, oo) for some a < M + 1. In particular, there are only a finite number of equivalence 
classes. We summarize these results in the following definition. 

Definition 4.3. Let K + 1 be the number of equivalence classes of «-> on I . We denote by 
(k)i<i<K and (ri)i<i<K the left (resp. right) endpoints of the equivalence classes: 



[I 



k-x^k-x], [lK,r K ). 



• The equivalence classes of <-> on I are {0}, 

• < < n < h < i'2 < ■ ■ ■ < rn-i < Ik < Tk = oo. 

• We have l K < M +1, 

We denote by 1 < k < K) the sub-matrices of P defined by Pk = (p(i, j))i k <i,j<r k ■ 
construction, the (Pk) are irreducible sub- stochastic matrices and P has the form 

I p~| o ... o 



By 



P = 



Pi 







Po 



Pk 

(.nfinite class) ' 

Remark 4.4. The sequences [k)i<i<K and {ri)i<i<K-i can be explicitly expressed in terms 
of the positions of the zeros in the vector (pi, . . . ,Pm)- By construction, we have 

{k, 1 < i < K} = {n > 1, p{n,n) > and p(n— l,n) = 0} 
{n, 1 < i < K - 1} = {n > l,p(n,n) > and p(n,n+ 1) = 0}, 

which we may rewrite in terms of the cookie vector: 

{k,l<i<K} = {n> l,jj{l < j < 2n- l,pj = 0} = n- 1 and p 2 n-i + 0} 
{r 4 , 1 < i < X - 1} = {n > < j < In- l,pj = 0} = n- 1 andp 2n = 0}. 

For example, if there is no cookie with strength 0, then K = 1 and h = 1. Conversely, if all 
the pi's have strength (the digging random walk case), then K = 1 and h = M + 1. 
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4.2 The process Z 



In order to study the branching Markov chain L introduced in the previous section, it is 
convenient to keep track of the typical evolution of a particle of L: fix a deterministic 
sequence (ji)i>o G {1, . . . , b} K and set 

def 

Xq = o, 

Xi+\ = Xi H for i > 0. 
Define the process Z = (Z n )„>o by 

Z n — i.{x n ). 

According to (c) of Lemma [3.2[ given a particle x located at l(x), the positions of its b 
children have the same law. Therefore, the law of Z does not depend on the choice of the 
sequence (ji)i>o- Moreover, Lemma I5~2l yields : 

Lemma 4.5. Under Pi, the process Z is a Markov chain starting from i, with transition 
matrix P given in Definition ] 3. 11 

Let us note that, if Z n is in some irreducible class [h,rk], it follows from Lemma |4~T1 
that Z m < r-fc for all m > n. Thus, Z can only move from an irreducible class [h,rk] to 
another class [lk',i~k'} where k' < k. Recall also that {0} is always an irreducible class (it is 
the unique absorbing state for Z). We introduce the absorption time 

To = inf{fc > 0, Z k = 0}. (7) 

Lemma 4.6. Assume that the cookie environment is such that q < 6/(6+1). Let io € IN, 
we have 

(a) T < oo Pi -a.s. 

(b) For any a > 0, sup„ E.; [Z"] < oo. 

Proof. The proof of the lemma is based on a coupling argument. Recall Definition 13.11 
and notice that the sequence (£k)k>M+i is i.i.d. Thus, for any stopping time r such that 
t > M + 1 a.s., the number of random variables in the sub-sequence (£fc)fe>T taking value 1 
before the first failure in this sub-sequence has a geometric distribution with parameter 

s d = f PUm+i = 1 | W G {0, 1}} = — -i -. 

q+b(l- q) 

It follows that, for any i, the number of random variables in the sequence (£k)k>i taking 
value 1 before the i th failure is stochastically dominated by M + Q\ + . . . + Qi where (Gk)k>i 
denotes a sequence of i.i.d. random variables with geometric distribution i.e. 

P{Gk = n] = (1 - s)s n for n > 0. 

This exactly means that, conditionally on Z„ = i, the distribution of Z n+ \ is stochastically 
dominated by Gi + ■ ■ ■ + Qi + M. Let us therefore introduce a new Markov chain Z with 
transition probabilities 

P{Z n+1 = j | Z„ = i} = PiGi + . . . + Qi + M = j}, 

It follows from the stochastic domination stated above that we can construct both processes 
Z and Z on the same space in such way that, under P, , almost surely, 

Zo = Zo = io and Z„ < Z n for all n > 1. (8) 
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The process Z is a branching process with geometric reproduction and with M immigrants 
at each generation. Setting 

we get 

E[Z„+i | Z n ] = cZ n + M. (9) 

When q < b/(b + 1), we have c < 1 so that Z n > M/(l - c) implies E[Z n+1 \ Z n ] < Z n . 
Therefore, the process Z stopped at its first hitting time of [0, Mj (1 — c)} is a positive super- 
martingale which converges almost surely. Since no state in [Mj (1 — c), oo) is absorbent for 
Z, we deduce that Z hits the set [0, M/ (1 — c)] in finite time. Using the Markov property of 
Z, it follows that Z returns below M/(l — c) infinitely often, almost surely. Since Z < Z , the 
same result also holds for Z. Furthermore, the process Z has a strictly positive probability 
of reaching from any i < M/(l — c) in one step (because no cookie has strength 1). Thus 
Z reaches in finite time. This entails (a). 

Concerning assertion (b), it suffices to prove the result for the process Z when a is 
an integer. We prove the result by induction on a. For a = 1, equation ([9]) implies 
E[Z„+i] = cE la [Z n ] + M so that 

supE. in [Z„] < max(i ,M/(l - c)). 

n 

Let us now assume that, for any (3 < a, E io [Z^] is uniformly bounded in n. We have 

3i [ Z n+i1 



E i0 [^+ X ] = V t0 [E{(Gi + ... + Gz n +Mr+ 1 \Z n }} 



= c Q + 1 E l0 K+ 1 ]+E s ; [Q(Z„)] (10) 

where Q is a polynomial of degree at most a. Therefore the induction hypothesis yields 
sup n |Ej [<2(iT n )]| < oo . In view of ifl0|) . we conclude that sup„ Ei [Z" +1 ] < oo. □ 

The following lemma roughly states that Z does not reach with a "big jump". 

Lemma 4.7. Assume that the cookie environment is such that q < b/(b + 1). Recall that 
[Ik, oo) denotes the unique infinite irreducible class of Z . We have 

inf P,-{3n > 0,Z n = l K } > 0. 
j>ik 

Proof. We introduce the stopping time 

cr = inf{n > 0, Z n < M + 1}. 

We are going to prove that 

inf P 1 {Z a = M + l}>0. (11) 

j>M+l J 

This will entail the lemma since Pm+i{^i = Ik} > (recall that Ik < M + 1). According 
to (a) of Lemma I4.6[ a is almost surely finite from any starting point j so we can write 

M+l oo 

i = p j{^-i = * and z ° = k } 

k=0 i=M+2 

M+l oo C h\ 

Let us for the time being admit that, for i > M + 1 and k g {0, . . . , M + 1}, 

M+l 

p(i,k)<(-) p(i,M+l). (13) 
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Then, combining l[T2"Tl and (jT3j) , we get 



6X M+1 



i=M+2 ^j=0 

(M + 2)P j {Z (T = A/ + 1}, 
which yields (jTTJ) . It remains to prove (f]~3)) . Recalling Definition 1 3. 1[ we have 

oo 

P (i,k)=J2 E p{a -e 1 ,...,e„ = e„}pu„+i = o}. 

n=M ei,...,e„ s.t. 

tJ{j<n,ej=l}=fc 
H{j<n,e 3 -=0}=i-l 

Keeping in mind that (£j)j>M+i are i.i.d. with P(£j = 1) = q/b, we get, for n > M, 

✓ , x Af+l-fc 

P{£n+1 = 0} - (-) P{£„+1 - 1, ... , tn+M+l-k = l}P{£„+M+2-fc = 0}. 

Thus, 

(, \ M+l-k oo 
~) E E PUi = ei,...,& = efl }PU fi+ i=0} 

^' n=M ei,...,es s.t. 

Hi<n,ej=l}=M+l 
K{j<n,e3=0}=i-1 



; (-) j»(*,M+l). 

□ 



5 Proof of Theorem 11.21 

The monotonicity result of Theorem 11.21 was proved in Corollary 13.81 It remains to prove 
the recurrence/transience criterion. The proof is split into four propositions: Proposition 

EH IH ES and EH 

Definition 5.1. Given an irreducible non negative matrix Q, its spectral radius is defined 
as: 

j_ 

A= lim U n \i,j)Y , 

n — >oo \ / 

where q^ n \i,j) denotes the coefficient of the matrix Q" . According to Vere- Jones ]2S^ . 
this quantity is well defined and is independent of i and j . 

When Q is a finite matrix, it follows from the classical Perron-Frobenius theory that A is 
the largest positive eigenvalue of Q. In particular, there exist left and right A-eigenvectors 
with positive coefficients. However, when Q is infinite, the situation is more complicated. 
In this case, one cannot ensure, without additional assumptions, the existence of left and 
right eigenvectors associated with the value A. Yet, we have the following characterization 
of A in terms of right sub-invariant vectors (c.f. [22j, p372): 

• A is the smallest value for which there exists a vector Y with strictly positive coefficients 
such that QY < XY. 
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By symmetry, we have a similar characterization with left sub-invariant vectors. Let us 
stress that, contrarily to the finite dimensional case, this characterization does not apply to 
super-invariant vectors: there may exist a strictly positive vector Y such that QY > X'Y 
for some A' > A. For more details, one can refer to |18[ 122] . 

Recall that, according to Definition 14,31 Pii ■ ■ ■ > Pk denote the irreducible sub-matrices 
of P. Let Ai, . . . , Xk stand for their associated spectral radii. We denote by A the largest 
spectral radius of these sub-matrices: 

A = max(Ai, . . . , Xk)- (14) 

5.1 Proof of recurrence 

Proposition 5.2. Assume that the cookie environment C — (pi, . . . ,pm ■ q) is such that 

b , , 1 

q < ana X < —. 

6+1 ~ b 

Then, the cookie random walk is recurrent. 

The proposition is based on the following lemma. 

Lemma 5.3. Let k G {1, . . . , K} and assume that A& < 1/6. Then, for any starting point 
£(o) = i G [If., rfc] and for any j G [Ik, we have 

${x G T, £(x) = j} < oo Pi-a.s, 

Proof of Proposition \5.2[ We assume that A < 1/6 and q < 6/(6+ 1). For k < K, the 
irreducible class [Zfc,rjt] is finite. Thus, Lemma [5731 insures that, for any i G 

%{x G T, £(x) e [l k , r k }} < oo P 4 -a.s. (15) 

We now show that this result also holds for the infinite class [Ik, oo) by using a contradiction 
argument. Let us suppose that, for some starting point £(o) = i, 

~Pi{${x G T, £{x) >l K } = oo} = c>Q. 

Then, for any n, 

Pi{3x G T, |x| > n and £(x) > l K } > c. (16) 

According to Lemma \4.7\ given a particle x located at £{x) = j > Ik, the probability that 
one of its descendants reaches level Ik is bounded away from uniformly in j. In view of 
(fl6|) . we deduce that, for some constant d > 0, uniformly in n, 

Pi{3x G T, \x\ > n and £(x) = l K } > c. 

This contradicts Lemma [5751 stating that 

tt{x G T, £(x) = l K } < oo Pi-a.s. 

Thus lfl5|) holds also for the infinite class. 

We can now complete the proof of the proposition. According to Corollary 13.51 we just 
need to prove that the branching Markov chain L starting from £(o) = M+l dies out almost 
surely. In view of (fl5|) . the stopping time N — inf{n, \fx G T„ £{x) < Ik} where all the 
particle are located strictly below Ik is finite almost surely. Moreover, if a particle x is 
located at £(x) = i£ {tk-IiIk) (i.e. its position does not belong to an irreducible class), 
then, the positions of all its children £(x ),. . . ,£(x ) are strictly below i. Thus, at time 
N' = N + (Ik — Tk-i — 1), all the particles in the system are located in [0,rjf-i]- We can 
now repeat the same procedure with the irreducible class [Ik-i, 1"k—i]- Since there are only 
a finite number of irreducible classes, we conclude, by induction, that all the particles of L 
are at zero in finite time with probability 1. □ 
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Proof of Lemma XSlk Fix k < K and jo € [Ik, fife]- By irreducibility, if suffices to prove that 

#{x e T, £{x) = j } < oo P io -a.s. (17) 

Let us note that, when k ^= K, the class [Zk, r^] is finite. Thus, the process L restricted 
to [Zfc,rfc] (i.e. the process where all the particles leaving this class vanish) is a multi-type 
branching process with only a finite number of types. Using Theorem 7.1, Chapter II of [9], 
it follows that this process is subcritical (it has parameter p = Xkb < 1 with the notation 
of [9] and is clearly positive regular and non-singular) and thus it dies out almost surely, 
which implies l|17p . However, this argument does not apply when k = K. We now provide 
an argument working for any k. 

As already mentioned, Criterion I of Corollary 4.1 of [22] states that is the smallest 
value for which there exists a vector Y k — {yi k ,Vi k +i, ■ ■ •), with strictly positive coefficients 
such that 

P k Y k < X k Y k . 

For k j^z K, the inequality above is, in fact, an equality. Since < 1/6, we get 

P k Y k < iy fc . (18) 



Define the function / : M i— > IN by 

m 



Vi for h <i <r k 
otherwise. 



Recall the definition of the Markov chain Z, with transition matrix P, introduced in the 
previous section. It follows from l|18p that, for any i € [0, rk], 

E[f(Z 1 )\Z =i]<lf(i). (19) 

We now consider a process L — (L n , n> 0) obtained by a slight modification of the process 
L: 

• L — L i.e. l{p) = i(6) = jo- 

• Li = L\, 

• For n > 1, L n is a branching Markov chain with the same transition probabilities as 
L except at point jo which becomes an absorbing state without branching i.e when a 
particle x is located at £(x) — jo, then t(x ) — jo and £(x ) = ... = £(x ) = 0. 

Following |15| . we consider the process 

i£T„ 

together with the filtration T n = <j(£(x), x € T<„). Using lfT9|) . we have 

E j0 [M n+1 \r n \ = J2 nfC^ 1 )) + --- + fC^ b ))\e(x)]+ J2 ww) 
= b nm)\z = k}+ ww) 

x£T„, l(x)=kjtj xeT n ,l(x)=j Q 

< E /(^»)+ E /(^») 

- Mn- 
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Thus, M n is a non-negative super-martingale which converges almost surely towards some 
random variable Moo with 

E iG [M 00 ]<E io [M ] = /(7o). 

Let N(n) denote the number of particles of L located at site jo at time n. Since jo is an 
absorbing state for the branching Markov chain L, the sequence N(n) is non-decreasing 
and thus converges almost surely to some random variable N(oo), Moreover, we have 
N(ri)f(jo) < M n so that iV(oo)/(j ) < Moo- This shows that is almost surely fi- 
nite and 

E JO [7V(oo)] <1. 

We can now complete the proof of the lemma. The random variable iV(oo) represents the 
total number of particles reaching level jo for the branching Markov chain L (where the 
particles returning at jo are frozen). Thus, the total number of particles reaching jo for the 
original branching Markov chain L, starting from one particle located at £(o) = jo, has the 
same law as the total progeny of a Galton- Watson process W = (W n ) n >o with Wo = 1 and 
with reproduction law iV(oc) (this corresponds to running the process L, then unfreezing all 
the particles at jo and then repeating this procedure). Thus, we get the following equality 
in law for the total number of particles located at jo for the original process L starting from 
one particle located at jo: 



b{x&TJ(x)=j } = Yw 



n=0 



Since E JO [iV(oo)] < 1 and P JO {iV(oo) = 1} < 1, the Galton- Watson process W dies out 
almost surely. This proves 



□ 



${x e T, (.(x) = j } < oo Pj -a.S. 
5.2 Proof of positive recurrence 

Proposition 5.4. Assume that the cookie environment C — (pi, . . . ,pm ', q) is such that 

b J x 1 

q < and A < — . 

6+1 b 

Then, all the return times of the walk to the root of the tree have finite expectation. 

Proof. Let Oi denote the time of the i th crossing of the edge joining the root of the tree to 
itself for the cookie random walk: 



o l = inf |n > 0, Y l{x 3 =x 3 _!=o} = 



We prove that E[cr,] < oo for all i. Recalling the construction of the branching Markov chain 
L in section [3TT1 and the definition of Z, we have 

oo 

E[oj] = i + 2Ej [ Yl i(x)]=i + 2j2b n 'Ei[Zn]- 

xeT\{o} n=l 

Let us for the time being admit that 

limsupP 4 {Z„ > 0} 1/n < X for any i. (20) 
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Then, using Holder's inequality and (b) of Lemma l4~6l choosing a,(3,\ such that A > A, 
6A 1 /" < 1 and £ + % = 1, we get 

OO OO oo 

Y, b n V t [Z n ] < b n P l {Z n > 0} 1 / a E i [Z*] 1/f> < Cp J2( b ~ Xl/a ) n < °°- 

n—1 n—1 n—1 

It remains to prove (|20|) . Recall that {0}, [h, r{\, . . . , [l^, oo) denote the irreducible classes 
of P and that Z can only move from a class [Zfc,rjfe] to another class [lk',T k '] with k' < k. 
Thus, for i G [lk,i"k], we have 

Pi{Z n > l k } = P t {z n e [l k ,r k ]} = p dz n = j} = Y P (n H*J)- 

For k < K, the sum above is taken over a finite set. Recalling the definition of A&, we get 
lim P,{Z n > l k } 1/n = A fc for all i £ [l k ,r k ]. 

n — >oo 

Using the Markov property of Z, we conclude by induction that, for any i < Ik, 

limsupPi{Z n >0} 1/n <max(Ai,...,Aif_i) < A. (21) 



It remains to prove the result for i> Ik- In view of Ipljl and using the Markov property of 
Z, it is sufficient to show that, for i> Ik, 

limsupPi{Z n > l K } 1/n < Xk- (22) 

n— >oo 

Let us fix i > Ik- We write 

oo 

Pi{Z n > l K } = P l {3m > n, Z m = l K ) + p 4^« = j}~Pj{M > 0, Z m = l K }- 

According to lemma l4~7l there exists c > such that, for all j > Ik, Pj{$m > 0, Z m = 
Ik} < 1 — c. Therefore, we deduce that 

1 1 °° 

P l {Z n > l K } < -P l {3m > n, Z m = l K } < - V p (m) (i, l K ). (23) 

r r * — » 



C 

m—n 



Moreover, we have limm^oo (p^ m ^ (i, Ik)) 1 ^" 1 = Ak < 1 hence 



o \ 1 /« 

lim ( V p W (ii y =Xk . (24) 



n — >oo 



\m—n 



The combination of ([23]) and l(24j) yields 112211 which completes the proof of the proposition. 

□ 

5.3 Proof of transience when A > 1/b 

Proposition 5.5. Assume that the cookie environment C — (pi, . . . ,pm ■ <?) is such that 
Then, the cookie random walk is transient. 
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Proof. The proof uses the idea of "seed" as explained in [16]: we can find a restriction L of 
L to a finite interval [I, r] which already has a non zero probability of survival. 

To this end, let us first note that we can always find a finite irreducible sub-matrix 
Q = (p(i, j))i<i,j< r of P with spectral radius A strictly larger than 1/6. Indeed, by definition 
of A, either 

• There exists k < K — 1 such that Afe > 1/6 in which case we set I = Ik and r = r^. 



• Otherwise \k > 1/6. In this case, we choose I = Ik and r > I. Lemma T4.2I insures 
that the sub-matrix Q = (p(i, j))i<i,j< r is irreducible. Moreover, as r goes to infinity, 
the spectral radius of Q tends to Xk (c./. Theorem 6.8 of [18]). Thus, we can choose 
r large enough such that the spectral radius A of Q is strictly larger than 1/6. 

We now consider the process L obtained from L by removing all the particles x whose 
position £(x) is not in [/, r] (we also remove from the process all the descendants of such a 
particle). The process L obtained in this way is a multi-type branching process with only 
finite number of types indexed by [l,r\. It follows from the irreducibility of Q that, with the 
terminology of [9], this process is positive regular. It is also clearly non singular. Moreover, 
the matrix M defined in Definition 4.1, Chapter II of [9], is, in our setting, equal to bQ 
so that the critical parameter p of Theorem 7.1, Chapter II of [9] is given by p = 6A > 1. 
Thus, Theorem 7.1 states that there exists i € [l,r] such that the process L starting from 
one particle located at position (i.e. with type) i has a non zero probability of survival. 
A fortiori, this implies that L also has a positive probability of survival. Thus the cookie 
random walk in transient. □ 

5.4 Proof transience when q > b/(b + 1) 

Proposition 5.6. Assume that the cookie environment C — (p±, . . . ,pm ', <?) is such that 

b 



(1 



> 



6+1' 



Then, the cookie random walk is transient. 

Remark 5.7. Under the stronger assumption q > 6/(6+ 1), one can prove, using a similar 
coupling argument as in the proof of Lemma I4.6( that the absorbtion time T of Z defined 
in ([7]) is infinite with strictly positive probability. This fact implies the transience of the 
cookie random walk. However, when q — 6/(6+ 1), the absorbtion time To may, or may not, 
depending on the cookie environment, be finite almost surely. Yet, Proposition 15.61 states 
that the walk is still transient in both cases. 



Proof of Proposition \5.6[ In view of the monotonicity property of the walk w.r.t. the cookie 
environment stated in Corollary I3.8( we just need to prove that, for any M, we can find 
q < b/(b + 1) such that the walk in the cookie environment 

C = (0,...,0;q) (25) 



is transient. It easily checked that the irreducible classes of the matrix P associated to 
a cookie environment of the form ([25]) are {0} and [M + l,oo[ (see, for instance, Remark 
4.4|) . Moreover, for such a cookie environment, the coefficients of P have a particularly 
simple form. Indeed, recalling Definition 13.1) a few line of elementary calculus yields, for 
i,j>M + l, 

p(i,j)=( j + i ~ M ~ 1 )sHl-s) i - M where , d = r —-f— (26) 
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(this result is proved, in a more general setting, in Lemma l7T3|) . Therefore, the polynomial 
vector U = - 1) . . . (i - M)) i>m+i is a right eigenvector of the irreducible sub-matrix 
Pi = (p(i, j))i.j>M+i associated with the eigenvalue 

i.e. P X U = XU. Similarly, setting V = ((s/(l - s)) i " 1 ) 4>M+1 , it also follows from ((26J) 

that V is a left eigenvector of P\ associated with the same eig envalue A i.e. t VP 1 = X t V. 
Moreover, the inner product VU is finite. Thus, according to Criterion III p375 of |22j . 
the spectral radius of P\ is equal to A. Since A tends to 1 as q increases to 6/ (6+1), we can 
find q < 6/(6 + 1) such that A > 1/6. Proposition 15.51 insures that, for this choice of q, the 
cookie random walk is transient. □ 



6 Rate of growth of the walk. 

6.1 Law of large numbers and central limit theorem 

We now prove Theorem 11.31 Thus, in rest of this section, we assume that X is a transient 
cookie random walk in an environment C = (pi , . . . , pm q) such that 

Pi > for alH G {1, . . . , M}. (27) 

The proof is based on the classical decomposition of the walk using the regeneration structure 
provided by the existence of cut times for the walk. Recall that T x denotes the sub-tree of 
T rooted at site x. We say that (random) time C > is a cut time for the cookie random 
walk X if it is such that: 

X, i T x ° for all i < C, 
Xi e T Xc for all i > G. 

i.e. C is a time where the walk first enters new subtree of T and never exits it. Let now 
(C„) n >i denote the increasing enumeration of these cut times: 

{C\ inf{fc > 0, k is a cut time}, 
C n +i = f infjfc > C„, k is a cut time}, 

with the convention that inf{0} = oo and C n +\ = oo when C n = oo. 

Proposition 6.1. Suppose that the sequence of cut times {C n ) n >\ is well defined (i.e. finite 
a.s.). Suppose further that E[Cj] < oo. Then, there exist deterministic v,a > such that 

\X„\ a . s . \X n \ ~nv , 2 
ana = — > AI(v,<t ). 



n n— >oo 



Proof. Let us first note that the event A = {X never crosses the edge from o to o} has non 
zero probability since the walk is transient and no cookies have strength (in this case, the 
irreducible classes for the matrix P are {0} and [l,oo)). Recalling that the walk evolves 
independently on distinct subtrees, it is easily seen that the sequence (C n +i —C n , \Xc n+1 \ — 
\Xc n \)n>i is i.i.d. and distributed as (C\, \Xc±\) under the conditional measure P{-|A} (c.f. 
for instance [3 [10] for details). Since P{^4} > and the walk X is nearest neighbor, we get 
E[(C7 n+1 - C n ) 2 } = E[Cl\A] < oo and E[(|X C „ +1 | - \X C J) 2 ] = E[|X Cl | 2 \A] < oo. Thus, 
we have 

^^nCAA], I^^E[|X Cl P], \XcA-n\XcA\A\n ^ ?m 
and the proposition follows from a change of time, c.f. [2[l0] for details. □ 
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Theorem 1 1.31 will now follow from Proposition ^. II once we have shown that the cut times 
of the walk are well defined and have a finite second moment. We shall, in fact, prove the 
stronger result: 

Proposition 6.2. The cut times of the walk are well defined and, for all (3 > 0, E[Cf ] < oo. 

The proof of this result relies on the following two lemmas whose proofs are provided 
after the proof of the proposition. 

Lemma 6.3. Recall the definition of the branching Markov chain L. Let U denote the total 
number of particles not located at for the entire lifetime of the process i.e. 

U = ${x e T, t{x) > 0}. 

There exists c\ > such that, for all n, 

Pi{C7 > n | L dies out} < Cl e^™ 1/3 . 

Lemma 6.4. Let (j n )n>a denote the increasing sequence of times where the walk visits a 
new site: 

def n 

7„+i = inf{fc > 7„, X k ^ X, for all i < k}. 
There exist v, C2 > such that, for all n, 

P{7n > n»} < c 2 e- n . 

Proof of Proposition \6.2[ We need to introduce some notation. We define two interlaced 
sequences (Si)j>o and (A)i>o by 

j S = 71, 

| D = inf{n > S , X n =X So = o}, 
and by induction, for k > 1, 

J Sk = inf{7„, 7„ > D k -i}, 
\ D k = inf{n>S k , X n =X Sk }. 

with the convention that, if Du — oo, then Dj, Sj — oo for all j > k. Let us set 

X = inf{fc>0,^ fe = oo}. 

Since the walk visits infinitely many distinct vertices, we have S k < oo whenever D k -i < oo 
so that these two interlaced sequences have the form 

So < D < Si < Di < . . . < S x < D x — oo. 

The interval [Sk, D k ) represents the times where the walk performs an excursion away from 
the set of vertices it has already visited before time S k . With these notations, the first cut 
time is given by 

Ci = S x . 

For n, m such that X m G T Xn , we use the slight abuse of notation X m — X n to denote 
the position of X m shifted by X n i.e. the position of X m with respect to the subtree T Xn . 
Using the Markov property for the stopping times Sk, Dk and noticing that the walk evolves 
on distinct subtrees on the time intervals [Sk,Dk), it follows that (compare with Lemma 3 
of [7] for details): 
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(a) Conditionally on D k < oo (i.e. x > k), the sequences ((Xs^+i—Xs^oKiKDj-Sj, j < k) 
are i.i.d. and distributed as (Xi)i<D under the conditional measure P{-|-D < 00} with 
D = M{k>l,X k - 1 = X k = o}, 

(b) Conditionally on D k < 00, the random variable D k+ \ — Sk+i has the same distribution 
as D — So. In particular, P{D k+ i < 00 \ D k < 00} = P{D < 00}. Thus, x nas a 
geometric distribution with parameter r = P{Dq < 00} = P\{L dies out} > 0: 

P{X = k} = (1 - r)r fc for & > 0. 

Fact (b) implies, in particular, that the first cut time C\ = S x (and thus all cut times) is 
finite almost surely. It remains to bound the moments of C± . We write 

P{S X > n v } = P{S X > n" and x > alnn} + P{S X > n v and x < a Inn} 
< (1 - r )r alnn + P{S X >n u and X < a\nn} 

where a > and where v is the constant of Lemma IOI Let (3 > be fixed, we can choose 
a large enough so that 

P{S X > n v } < n J +1)v + P{5 X > n v and X < ahin}. (28) 
It remains to find an upper bound for the second term. Let us first note that 

alnn 

P{S X >n"andx<a Inn} < ^ P{S k > n v and x > k}. (29) 

fc=0 

We introduce the sequence (V k )k>Q defined by 

Vfe = number of distinct vertices visited by the walk during the excursion [S k , -Dfc), 

with the convention that V k = 00 when D k — 00. By definition of S k ,D k , the total number 
of distinct vertices other than the root visited by the walk up to time S k is exactly the sum 
of the number of vertices visited in each excursion [Si, -Dj) (i < k) which is Vo + ■ ■ ■ + V k -\. 
Thus, S k is the time where the walk visits its (Vq + . . . + V k -i + 2) th new vertex. This yields 
the identity 

Sk = 7V r o+-+V*_i+2 

which holds for all k with the convention 700 = 00. Thus, we can rewrite the r.h.s. of (|29|) 

as 

a Inn a In n 

?{Sk > n v and x > k} = ^ ?{lv a +...+v k _ 1+ 2 > n v and Vi+. . -+V k -i < 00}. (30) 

fc=0 fc=0 

Each term on the r.h.s. of (|30| is bounded by 

P{7y 0+ ...+y fc _ 1+2 > n v and V\ + . . . + V k -i < 00} 

= P{7v +...+v fc _i+2 > n v and n < Vo + ■ ■ ■ + V k -i + 2 < 00} 
+ P{7v +...+v fc _ 1+ 2 > n v and V + . . . + V k -i + 2 < n} 

< P{n - 2 < V + . . . + V k -x < 00} + P{ 7 « > n v } { ' U 1 

< P{n - 2 < V + . . . + Vfc_i < 00} + c 2 e-™ 

where we used Lemma l6~4l for the last inequality. Let us note that, according to Fact (a), 
conditionally on {Vo + ... V k -i < 00} = {x > k}, the random variables (Vo, V\, . . . , V k -i) 
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are i.i.d. and have the same law as the number of vertices visited by the walk before the time 
D of its first jump from the root to the root under the conditional measure P{- | D < oo}. 
Recalling the construction of the branching Markov chain L described in Section [3l we 
see that this distribution is exactly that of the random variable U of Lemma 16,31 under 
the measure P = f Pi{- | L dies out}. Let now (J7j)i>o denote a sequence of i.i.d. random 
variables with the same distribution as U under P. For k < a Inn, we get 

P{n - 2 < V + . . . + Vfe_i < 00} < P{V Q + . . . + Vfc-i > n - 2 | V + . . . + V k -i < oo} 

= P{U + ... + U k -i >n-2} 

- ( n — 2 
< (ahxn)P<U > 



a Inn 



< Cl (alnn)exp^(^) 3 j (32) 
where we used Lemma [6751 for the last inequality. Combining (J28J) - ((32j , we conclude that 
P{5 X > n v } < * + c 2 (a lnn)e~" + ci(alnn) 2 exp [ — ( — t~ — ) 3 | < 



for all n large enough. This yields E[>S^] < oo. □ 

We now provide the proof of the lemmas. 
Proof of Lemma nnH Let §L n denote the number of particles not located at at time n: 

$L n A = \{x e T n , £(x) > 0}. 
Let also 6 stand for the lifetime of L: 

9 = inf{n, jji„ = 0} 

with the convention 9 = oo when L does not die out. Since no cookie has strength 0, 
the irreducible classes of P are {0} and [l,oo). Thus, the transience of the walk implies 
Pi{9 < oo} 6 (0, 1). Let H denote the maximal number of particles alive at the same time 
for the process L: 

H = SUpjJI/n. 

n 

It follows from the inequality U < HQ that 

Pi{C7 > n, 6 < oo} < Pi{# > y/n, 6 < oo} + P^H < y/n, 6 > y/n}. (33) 

The first term on the r.h.s of (|33|) is easy to bound. Recalling the monotonicity property of 
Proposition 13.6} we have Pj{0 < oo} < Pi{6 < oo} for any j > 1. Therefore, using the 
Markov property of L with the stopping time C = infjfc, jj{a; € Tfc, £(x) > 0} > y/n}, we 
get, with obvious notation, 

Pi{H > V^,e < oo} = Ei[l {c<oo} P Lc {9 < oo}] 

— ^IV"! particles loc. at l{© < °°} 

= Pi{6 < oo}Lv^J 

< e-™ 1/3 (34) 

where the last inequality hold for n large enough. We now compute an upper bound for the 
second term on the r.h.s. of l(33|). Given k < y/n, it follows again from Proposition 13.61 that. 

Pl{ff < y/E, 9 > y/H) < Pi{$L k < VH} L " /(fe+1)J . (35) 
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(this bound is obtained by considering the process where all the particles at time (k + 
l),2(fc + 1), . . . are replaced by a single particle located at 1). Let us for the time being 
admit that there exist p > 1 and a > such that, 

a = liminf Pi{ttLi > ap 1 } > 0. (36) 

i — >oo 

Then, choosing k = [lnn/lnpj, the combination of (|35| and l|36p yields, for all n large 
enough, 

Pi{ff < V^,e > ^} < Pi{tfL fc < ap fe }L"/( fc+1 )J < (1 - a)L»/(fe+i)J < e -» 1/3 . (37) 
Putting <[33j) , ((34]) and ((3TJ) together, we conclude that, 

^ r rT 1 . ~P\{U > n, < 00} 1/3 

Pi{U >n\L dies out} = „ ~ ' , < c x e~ n 

PijB < 00} 

which is the claim of the Lemma. It remains to prove |36|) . Recall that q represents the 
bias of the walk when all the cookie have been eaten. We consider separately the two cases 
q < b/(b+l) and q > 6/(6+1). 

(a) q < b/(b + 1). Since the walk is transient, the spectral radius of the irreducible class 
[1, 00) of the matrix P is necessarily strictly larger than 1/6 (otherwise the walk would be 
recurrent according to Proposition I5.2|) . Using exactly the same arguments as in the proof 
of Proposition 15.51 we can nn d r large enough such that the finite sub-matrix (Pi,j)i<%,j<r is 
irreducible with spectral radius A strictly larger than 1/6. We consider again the process L 
obtained from L by removing all the particles x (along with their progeny) whose position 
i(x) is not in [1, r]. As already noticed in the proof of Proposition I5.5[ the process £ is a 
positive regular, non singular, multi-type branching process with a finite number of types 
and with parameter p = bX > 1. Therefore, Theorem 1 pl92 of [3J implies that, for a > 
small enough, 

lim Pi-fltij > ap 1 } > 

i — >oo 

which, in turn, implies <[36|) . 

(b) q > b/(b + 1). The spectral radius A of the irreducible class [1, 00) may, in this case, be 
strictly smaller that 1/6 (see the remark below the statement of Theorem 1 1.2|) . However, as 
shown during the proof of Proposition [521 we can always find q < 6/(6+ 1) < q such that the 
walk in the cookie environment (0, . . . , ; q) is transient. Therefore, the walk in the cookie 
environment C = (pi, . . . ,Pm ', <?) < C is also transient. Denoting by L the branching Markov 
chain associated with C, it follows from the previous case (a) combined with Proposition 13. 71 
that, for some p > 1, a > 0, 

liminf Pi{tJLi > ap 1 } > liminf Pi > ap 1 } > 0. 

□ 

Proof of Lemma [6m Recall that, given x G T and i £ {0, . . . , 6}, we denote by x the i th 
child of x (with the convention x =x). We call (un-rooted) path of length k an element 
[v\, . . . , Vk] € {0, . . . , b} k . Such a path is said to be increasing if V{ ^ for all 1 < i < k. 
Given x G T, we use the notation x[v%, . . . , Vk] to denote the endpoint of the path rooted at 

site x i.e. >v 

a;[0] = x and x[vi, . . . ,v k ]= x[v u . . . ,v k _{\ . 

The proof of the lemma is based on the following observation: given two increasing paths 
v, w with same length k such that [vi, . . . ,Vk] +~ [wt,..., Wk], we have 

x[vi, ...,Vk]j^ y[w\, ■■■,Wk\ for any x,y eT. 
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Let (iik)k>i denote the sequence of random variables taking values in {0, ... , b} defined by 
X n =X„_i- With the previous notation, we have, for any m < n, 

X n = X m [u m+ i . . . u n \. 

It follows from the previous remark that, for any fixed k < n, the number of distinct vertices 
visited by the walk X up to time n is larger than the number of distinct increasing sub- 
paths of length k in the random path [ui, . . . , u n ]. We get a lower bound for the number 
of such sub-paths using a coupling argument. Recall that no cookie has strength and set 
r/ = mini Pi/b > 0. It is clear from the definition of the transition probabilities of the cookie 
random walk X that 

f P{u„ = i\ u u . ..u n } >i] for i G {1,. . . ,b}, 

\ P{u n = | iti, . . .u n } < 1 - brj. 

Therefore, we can construct on the same probability space a sequence of i.i.d random vari- 
ables (u„)„>i with distribution: 

f P{u„ = i} = J] for i e {1, . . . , b} 

\ P{u n = 0} = 1 - bq, 

in such way that 

u n = i 7^ implies u n = i. 

With this construction, any increasing sub-path of [u\, . . . , u n ] is also an increasing sub-path 
of [iii, . . . ,u n ]. Moreover, since the sequence {u n )n>i is i.i.d., we have, for any increasing 
path [v!,...,Vk], 

P {[«!,... , u n ] does not contain the sub-path[wi, . . . , Vk]} 

Ln/feJ 

< [] P{[« --i)fc+i> ■■■.«;*] + [vi,.- .,v k ]} = (l- J? fe )L«AJ. (38) 

We now choose k = f [c In nj +1 with c = f 31n( - 1 1 / T) - ) and set <5 = f c In b. Since there are b k > n s 
increasing paths of length k, we get, for n large enough, 

P {[ui, • • ■ ,u n ] contains less than n s distinct increasing sub-paths of same length} 

< P {[iii, ■ ■ ■ , «„] does not contain all increasing sub-paths of length k}} 

<b k (l- ?? fe )L"/fcJ 

Thus, if V n denotes the number of distinct vertices visited by the cookie random walk X up 
to time n, we have proved the lower bound: 

P{V„ < n s } < e-^. 

Choosing v > max(l/5, 2), we conclude that, for all n large enough, 

P{7« > n v } = P{V Ln ,j <n}< P{V L „,j < [n» \ s } < < e~ n . 

□ 
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6.2 Example of a transient walk with sub-linear growth 

In this section, we prove Proposition II .91 whose statement is repeated below. 

Proposition 6.5. Let X be a C = (pi,P2> 0, ; q) cookie random walk with q > b/(b+ 1) 
and pi,P2 > such that the largest positive eigenvalue of the matrix 

/EL J- P1P2 _ 2p!p 2 P1P2 

p drf / b -t- b —gr~ -p~ 

1 1 I P1+P2 _ 2pip 2 P1P2 



6 2 



b' 2 



is equal to l/b (such a choice of p\,p% exists for any b > 2). Then, X is transient (since 
q> W(b+1)) yet 

liminf \M =0 . 



Proof. For this particular cookie environment, it is easily seen that the cookie environment 
matrix P has three irreducible classes {0}, [1, 2], [3, oo) and takes the form 



P : 



1 








Pi 







* 


* 






(infinite class) / 



V 

where P\ is the matric given in the proposition. By hypothesis, the spectral radius of 
the irreducible class [1,2] is l/b, therefore, the branching Markov chain L starting from 
£(o) € [1,2] dies out almost surely (the restriction of L to [1,2] is simply a critical 2-type 
branching process where each particle gives birth to, at most, 2 children). In particular, the 
quantity 

is Pj almost surely finite for i G {1,2}. Moreover, one can exactly compute the generating 
functions E,[s A ] for i € {1,2} using the recursion relation given by the branching structure 
of L. After a few lines of elementary (but tedious) calculus and using a classical Tauberian 
theorem, we get the following estimate on the tail distribution of A: 

Pi{A > *} ~ 4=, 

v x 

for some constant C > depending on pi , p 2 (alternatively, one can invoke Theorem 1 of [8] 
for the total progeny of a general critical multi-type branching process combined with the 
characterization of the domain of attraction to a stable law) . 

As in the previous section, let {^ n )n>o denote the increasing sequence of times where 
the walk visits a new site (as defined in Lemma l6^4|) and define two interlaced sequences 
(Si)i>o and (Di)i>o in a similar way as in the proof of Proposition 16. 21 (only the initialization 
changes) : 

' S Q = 

D = mf{n > 0, X n = A„_i = o}, 



and by induction, for k > 1, 



S k = inf{7„, 7„ > D fc _i}, 
D k =M{n > S k , X n =X Sk }. 
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Since, L starting from £(0) = 1 dies out almost surely, the walk crosses the edge from 
the root to the root at least once almost surely. Therefore, Do is almost surely finite. 
Using the independence of the cookie random walk on distinct subtrees, it follows that the 
random variables sequences (Si)i>o, (£>i)i>o are all finite almost surely. Moreover, recalling 
the construction of L, it also follows that the sequence of excursion lengths (-Rj)j>o = 
(Di — Si)i>o is a sequence of i.i.d. random variables, distributed as the random variable 
2A — 1 under Pi. We also have the trivial facts: 



Forallfc>0, \X Dk \ = \X Sk \ - 1. 

The walk only visits new vertices of the tree during the time intervals ([Sk, D k ]) k > . 



Thus, the number of vertices visited by the walk at time Sk is smaller than l+J2i=o R 



In particular, we have \Xs k \ < 1 + $3< = o R' 

• Forallfc>0, D k >J2- = o R *- 
Combining these three points, we deduce that, for k > 1, 

\ X D k | < J2i=0 R _ i R k 



Dk Ei=o R T,t=Q R 

Since, (Ri)i>o is a sequence of i.i.d random variables in the domain of normal attraction of 
a positive stable distribution of index 1/2, it is well known (and easily to check) that 

r Rk i 
limsup — T = 1 a.s. 

fc ^°° J2 l= Ri 



which, in turn, implies 



l im in f \JSA < lim in f ]*°A = q a . s . 

n— >oo n fe^oo Dk 



□ 



7 Computation of the spectral radius 

In this section, we prove Theorem ll.4l and Proposition ll.8l bv computing the maximal spectral 
radius A of the cookie environment matrix P. Recall that the irreducible classes of P 
are {0}, [Zi,ri], . . . , [tk, oo) and that P k denotes the restriction of P to [Ik, r k ] ([Ik, oo) for 
k — K). Denoting by \ k the spectral radius of P k , we have, by definition: 

A = max(Ai, . . . , \k)- 

Since the non negative matrices Pi, ... , Pk-i are finite, their spectral radii are equal to their 
largest eigenvalue. Finding the spectral radius of the infinite matrix Pk is more complicated. 
We shall make use on the following result. 

Proposition 7.1. Let Q — (q(i, j))i,j>i be an infinite irreducible non negative matrix. 
Suppose that there exists a non-negative left eigenvector Y — (y,)i>i of Q associated with 
some eigenvalue v > i.e. 

*yg = ^*y. (39) 

Assume further that, for all e > 0, there exists N > 1 such that the finite sub-matrix 
Qn — (q(i, j))i<i,j<N is irreducible and the sub-vector Yn — (yi)i<i<N is v — e super- 
invariant i.e 

l Y N Q N > (v-eYY N . (40) 
Then, the spectral radius of Q is equal to v. 
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Remark 7.2. By symmetry, the proposition above remains unchanged if one considers a 
right eigenvector in place of a left eigenvector. Let us also note that Proposition 17.11 does 
not cover all possible cases. Indeed, contrarily to the finite case, there exist infinite non 
negative irreducible matrices for which there is no eigenvector Y satisfying Proposition 17. II 

Proof. On the one hand, according to Criterion I of Corollary 4.1 of [22], the spectral radius 
Aq of Q is the smallest value for which there exists a non negative vector Y ^ such that 

l YQ < A Q *Y. 

Therefore, we deduce from f39|) that 

v > A Q . 

On the other hand, the matrix Qn is finite so that, according to the Perron- Frobenius 
Theorem, its spectral radius is equal to its largest eigenvalue Xq n and is given by the 
formula 

X Qn = sup mm — 

(xi,...,x N ) 3 x j 

where the supremum is taken over all iV-dimensional vectors with strictly positive coefficients 
(c./. (1.1) p.4 of [18]). In view of flD]). we deduce that Aq n >v — e, 

Furthermore, when Qn is irreducible, Theorem 6.8 of [18] states that Aq n < Aq. We 
conclude that 

Aq < v < X Q + e. 

□ 



7.1 Preliminaries 

Recall the construction of the random variables (£i)i>i given in Definition [3J] and set 

£ m ,n = {in the finite sequence £2, ■ ■ ■ , £m ), there are at least m terms equal to 

and exactly n terms are equal to 1 before the m th 0} 
£'m n { m the finite sequence (d, £2, • • ■ , Cm), there are exactly m terms equal to 

and exactly n terms equal to 1 } . 

Let us note that, for n + m > M, 

P{£ m ,n] - P{4,») = 0. (41) 

In the rest of this section, we use the notation 

s = q+{ l_ q)b = p {^+i = 1 I e {°» !}}• 

Lemma 7.3. Fori,j > 1, the coefficient p(i, j) of the matrix P associated with the cookie 
environment C = (pi, . . . ,pm ', q) is given by 

0<n<j \ J J 

0<m<i-l 

Proof. Recall that p(i,j) is equal to the probability of having j times 1 in the sequence 
(&)(>i before the i th 0. We decompose this event according to the number of 0's and l's in 
the subsequence (£i)i<m- Let T m ,n be the event 

J-m,n = {in the sub-sequence (£i)j>Mi n terms equal to 1 before the m th failure}. 
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Thus we have 

P (i,j)=P{£ i , j }+ J2 P {C»}P{^-m,i-n} ( 42 ) 

0<n<j 
0<m<i-l 

(the first term of the r.h.s. of the equation comes from the case to = i which cannot be 
included in the sum). Since the sequence (£i)i>M is a sequence of i.i.d. random variables, it 
is easy to compute P{jF mj „}. Indeed, noticing that, 

P{F„ hn } = P{^ m ,„ | $ G {0, 1} for all I G [M, M + n + to]}, 

we get 

P{F m ,n}= (^^"^(l-sr. (43) 

The combination of lf42j) and lf43j) completes the proof of the lemma. □ 

We can now compute the image YP of the exponential vector Y = ((s/(l — s))* -1 )i>i. 
Let us first recall the notation 



, def __q_ tt ( n \ ( q \ (b-l)Pi , v% 



We use the convention that = when u > v. 
Lemma 7.4. We have 



oo / \ i— 1 / \ j— 1 



M-j / vi-l M M-n / s j+m-n 



au) d =< E ptfw) E E p KvJ ( r 

i— 1 ^ ' n=j + l m— 

7n particular, A(J) — /or j > M. 

Proof. With the help of Lemma [773} and in view of (j41|), we have 

oo / _ \ i — 1 



00 / \ i—1 00 

\ 1 — m 



E p fe}(^) +E E p^jf^^r'y^c 1 -) 1 

i=l V / i=l 0<n<j V 7 

0<m<i-l 

Af-j / x i-1 



i=l 0<ii<j'AM i=0 

0<rn<M-n 



= E p {^}(i^j + E p{^.«> E ( T j ^ +i+m - n (i-«) 

Using the relation 



1 — m 



L _ , N j+m—n 



i=0 

we deduce that 



i I V 1 — s 



i-1 M-j . x i-1 jAM M-n , x j+m-n 

S \ \ - X - , , / S x 



(ifj ^ E p i^}(i^) +E E p {C,j( T 

i=l v 7 i=l v 7 n=0 m=0 

M M — n , s j+m—n 



n— rn— 
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It simply remains to show that 

M M-n / „ \ m-n+1 

A, 

Let us note that, 



l - = EE p {U ■ ( 44 ) 



M / 1 

A„ym = 737 II T^ P ^' = ^ + ^ ^ ^ + = 

;=i ^ 



Expanding the r.h.s. of this equation and using the definition of £' mn , we get (|44| which 
concludes the proof of the lemma. □ 

We have already noticed that A(j) = whenever j > M. In fact, if some cookies have 
strength 0, the lower bound on j can be improved. Let Mq denote the number of cookies 
with strength 0: 

M o = jJ{l<i<M, Pi = 0}. 

Lemma 7.5. Let C = (pi, . . . ,pm ',q) be a cookie environment with pu ^ 0. We have, 

A{j) = for all j > M - Mq. 

Proof. Since Mq cookies have strength 0, there are at most M — Mq terms equal to 1 in the 
sequence . . . , £,m)- Keeping in mind the definitions of £ m ,n and £' mn , we see that 

P{£ m ,n} = P{£' m , n } = for n > M - Mq. 

Moreover, recall that pm ^ 0. Thus, if exactly M — Mq terms are equal to 1, the last one, 
£mj must also be equal to 1. Therefore, we have 

P{£ m .M-M } = 0. 

Let us now fix j > M — Mq, and look at the expression of A(j). 

M-j , > t-1 M M-n , ^ j+m-n 

AU) d = f E p {^} (t^J - E E p {£LJ 

i=l ^ ' n=j+l m=0 ^ 

The terms in the first sum YlfLi'' are a U zero smce 3 > M — Mq. Similarly, all the terms in 
the sum J2m=o are a ^ so zer0 smce n>j + l>M — Mq. □ 

Proposition 7.6. Let C = (pi, . . . ,pm ]Q) be a cookie environment such that 



b 

q < and Mq > 

o+l 



M 

T 



If M is an odd integer, assume further that pu 7^ 0. Then, the spectral radius \k of the 
infinite irreducible sub-matrix Pk = (p(i, j))i,j>i K * s equal to X sym . 

Proof. Let us note that, when M is an even integer and pu — 0, we can consider C as 
the M + 1 cookie environment (pi , . . . , pu >Q',q) and this M + 1 cookie environment still 
possesses, at least, half of its cookies with zero strength because [(M + 1)/2J = [M/2\. 
Thus, we can assume, without loss of generality that the cookie environment is such that 
Pm 0. In order to prove the proposition, we shall prove that 



Y = 



i>lf 
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is a left eigenvector of Pr for the eigenvalue A Bym fulfilling the assumptions of Proposition 
17.11 Since the cookie environment has Mo > \M/2\ cookies with strength 0, there are, in 
the 2\M/2\ first cookies, at most [M/2\ random variables taking value 1 in the sequence 
(&)»>i before the |M/2j th failure i.e. 

p(i,j) = for i < [M/2\ < j. (45) 

This implies, in particular, that Ik > [M/2\ + 1 > M — Mq. Using Lemma 1731 we deduce 
that 

A(j) = for all j > l k . (46) 
Combining lj46j) and Lemma EH we conclude that Y is indeed a left eigenvector: 

l YP K = A Bym l Y. 

Let e > 0. We consider the sub-vector 

i-l\ 



v iff 

IN — 



1 - S 



l K <i<l K +N 
'v. _ o \ / \ ~*\ *i 



It remains to show that, for N large enough, YnPk,n > (^k — e) X/v i.e. 



Ik+N-1 / x i-1 /x j-l 



E ^'^(i^J - (Aa ' _£ HT^/ for allje {/«-,..., + AT -1}. 

(47) 



s \ / s 



Keeping in mind that, for j > Ir 

oo 

we see that l(47j) is equivalent to proving that, 

S • • 



1 - s 



i=l K +N 



E - £ {T— S ) ioxj€{l K ,...,l K + N-l}. (48) 



Choosing TV such that Ik + N > M + 1, and using the expression of j) stated in Lemma 
7j3l we get, for any j € {l K , ...,Ir + N-1}, 



i=l K +N V ' 7 0<n<j i=i K +Ar \ J / 

0<m<Af 



where we used that P{£^ „} = P{£ m ,„} = when either n or m is strictly larger than M. 
We now write 



•j + i-m-n-l\ aJ . +i _ 1 _ B 



j -n 



j+m— n 



j + i — to — n — 1 



1 — s J V i — to — 1 

and we interpret the term 

j + i-m-n-l \ i- m -i^ _ s y-» 
i — to — 1 
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as the probability of having (i—m—1) successes before having (j—n+1) failures in a sequence 
(B r ) r >i of i.i.d. Bernoulli random variables with distribution P{B r = 1} = 1 — P{B r = 
0} = s. Therefore, we deduce that 



oo 

£ 

i=l K +N 



j + i — m — n 
i — m—1 



YJ-n+l 



= Pjthere are at least Ik + N — m— 1 successes before the (j — n+l) th failure in (i? r )r>i} 
< Pjthere are at least Ik+N— M— 1 successes before the (lK+N + l) th failure in (S r )r>i}- 

Noticing that s < 1/2 since q < b/(b + 1), the law of large numbers for the biased Bernoulli 
sequence (-B r ) r >i implies that the above probability converges to as iV tends to infinity. 
Thus, for all e > 0, we can find N > 1 such that (M holds. □ 



7.2 Proofs of Theorem 11.41 and Proposition 11.81 

Proof of Theorem \1.4\ Consider a cookie environment C — (pi,... ,Pm q) such that: 

b 



q < 



6 + 1 



and pi = for all i < [M/2\ . 



Recall that M = jj{l < i < M,pi — 0} stands for the number of cookies with strength 
0. We simply need to check that the irreducible classes of P are {0} and [M + 1, oo) i.e P 
takes the form: 



p = 



o \ 



o 







V 



and it will follows from Proposition 17.61 that A 
that: 



Ai 



Pi 

-- A.. 



/ 



Thus, we just need to check 



(1) for all 1 < i < Mq, p(i, i) = (the index i does not belong to any irreducible class). 

(2) for all j £ IN, p(M + > (Mo + 1 belongs to the infinite irreducible class). 

The second assertion is straightforward since there are only Mq cookies with strength 0. In 
order to see why (1) holds, we consider the two cases: 

• 1 < i < |_iV4"/ 2j . Then, clearly p(i,j) = l{ ;)= o}- I n particular p(i, i) = 0. 

• \M/2\ + 1 < i < Mq. In this case, there are Mq > i cookies with strength in the 
first 2i — l>M cookies. Therefore, there cannot be i random variables £ taking value 
1 in the sequence (£k)k>i before the i th failure. This means that p(i,i) = 0. 



□ 



Kti 



Proof of Proposition \1.8[ Let X be a (p\,p2, 0, . . . , ; q) cookie random walk with K > 2. 
Using similar argument as before, it is easily checked that the irreducible classes of the 
cookie environment matrix P are, in this case, {0}, [1,2] and [K + 1, oo). Moreover, the 
matrix associated with the irreducible class [1,2] is given, as in Proposition 16. 5} by 



£1 + 



P1P2 _ 2pip 2 

b h 1 

1+P2 _ 2pip 2 

b b 2 



I' IP 2 

i>a>i 
b 2 



(49) 
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P{X drift to 00} 




Figure 3: Phase transition of a (p,p;0) cookie random walk on a binary tree. 



Thus, denoting by v the largest spectral radius of this matrix and using Proposition 17.61 we 
deduce that, for q < the maximal spectral radius of P is given by: 

A = max(i/, A sym ). 

We conclude the proof of the Proposition using Theorem 11.21 and the fact that A aym > 1/6 
whenever q > (c.f. Remark [L5| . □ 

8 Other models 

8.1 The case q = 0. 

As stated in Proposition 11.81 a (pi,P2, 0, ; q) cookie random walk is transient as soon as 
the spectral radius v(pi,p2) of the matrix given in (|49j) is strictly larger that 1/6. Let us 
remark that this quantity does not depend on q. Therefore, when v(p\,p2) > \ the walk is 
transient for any arbitrarily small q. In fact, using similar arguments to those provided in 
this paper, one can deal with the case q = 0. The study of the walk is even simpler in this 
case since the cookie environment matrix P does not have an infinite class (p(i,i) = for 
all i > M). Thus, the process L is, in this case, just a multi-type branching process with 
finitely many types. 

However, when q = 0, a — 1 law does not hold for the walk anymore since it always has 
a strictly positive probability of getting stuck at o eventually (this probability is bounded 
below by — Pi)). Therefore, the recurrence/transience criterion now translates to finding 
whether the walk eventually gets stuck at o with probability 1 or has a positive probability 
of drifting towards infinity. 

For instance, an easy adaptation of Proposition 11.81 (the details are left over for the 
reader) shows that, for a two cookies environment C = [pi,p% ;0), the walk has a positive 
probability of drifting towards infinity if and only if v(p\ 1 p2) > t. Moreover, the process L 
is, in this setting, a 2-type branching process and the probability that the walk gets stuck at 
o is equal to the probability that L, starting from one particle located at 2, dies out. This 
probability of extinction is obtained by computing the fixed point of the generating function 
of L (c.f. Theorem 2, pl86 of [3]) and yields 

f 1 if v(pi,p 2 ) < i, 

P{X is stuck at o eventually} = <^ (i-p 1 ){b+b P2 + P 2 1 pl-b PlP 2 2 - PlP l-b PlP2 ) . f , , 1 

I P1P2O-1) 11 v \P^-iPV ^ b- 

An illustration of this phase transition is given in Figure [3] for the case of a binary tree. 
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8.2 Multi-excited random walks on Galton- Watson trees 



In the paper, we assumed that the tree T is regular. Yet, one may also consider a cookie 
random walk on more general kinds of trees like, for instance, Galton-Watson trees. Recalling 
the classical model of biased random walk on Galton- Waston trees [13j, a natural way to 
define the excited random walk on such a tree is as follows: a cookie random X on T in a 
cookie environment B — (Pi, ... , (3m ', ot) £ [0, oo) M x (0, oo) is a stochastic process moving 
according the following rule: 

• If X n — x is at a vertex with B children and there remain the cookies with strengths 
j3j, (3j+i, . . • , /3m at this vertex, then X eats the cookie with attached strength j3j and 
then jumps at time n + 1 to the father of x with probability ppga and to each son of 

x with probability y+bb ■ 

• If X n = x is at a vertex with B children and there is no remaining cookie at site x, 
then X jumps at time n + 1 to the father of x with probability 1+ 1 Ba and to each son 
of x with probability 1+ g a . 

In the case of a regular 6-ary tree, this model coincides with the one studied in this paper 
with the transformation pj = jqr|g- and q = b ^_ 1 ■ In this new setting, one can still construct 
a Markov process L associated with the local time process of the walk and one can easily 
adapt the proof of Theorem ll.2l to show the following result: 

Let X be a B = (/3i, . . . ,/3m \a) cookie random walk on a Galton-Watson tree T with 
reproduction law B such that ~P{B = 0} = and P{B = 1} < 1 and E[S] < oo. Fix b > 2 
and let P be the matrix of Deftnition \3.1\ associated with a cookie random walk on a regular 
b-ary tree in the cookie environment C — (pi, ■ ■ ■ ,Pm iO), where pt = EiT+l an ^ 1 ^ ba+i 
(this matrix does not, in fact, depend, on the choice ofb). Then, the B-cookie random walk 
on the Galton Watson tree T is transient if and only if 

a>l or A(C) > — , 

where X(C) denotes, as before, the largest spectral radius of the irreducible sub-matrices of 
P. 

Acknowledgments. We would like to thank Gady Kozma for stimulating discussion con- 
cerning this model. We would also like to thank Sebastian Miiller and Serguei Popov for 
showing us how to prove the recurrence of the walk in the critical case A(C) = 1/6. 
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